1. Field of the Invention
The subject invention relates to a method of determining alignment of images in high dimensional feature space, and more specifically, to determining alignment of a sequence of images of different modalities from various types of applications.
2. Description of the Prior Art
Registering images aligns a target image onto a source, or reference, image using various registration algorithms to enhance image correlation, remove geometric distortion, and to facilitate various image processing tasks, such as image analysis, segmentation, understanding, visualization, and rendering. Image registration methods select a sequence of intensity preserving transformations, or algorithms, to maximize an image similarity measure between the reference image and the target image. Image registration has wide applications in medical imaging, DNA sequencing, video motion analysis, satellite imagery, remote sensing, security and surveillance. The accuracy of the registration algorithm critically depends on two factors: the selection of a highly discriminating image feature space and the choice of similarity measure to match these image features. These factors are especially important when some of the intensity differences are due to the sensor itself, as arises in registration with different types of imaging sensors or registration of speckle-limited images.
Multi-sensor images typically have intensity maps that are unique to the sensors used to acquire them and a direct linear correlation between intensity maps may not exist. Several other matching functions have been suggested in D. Hill, P. Batchelor, M. Holden, and D. Hawkes, “Medical image registration,” Phys. Med. Biol., vol. 26, pp. R1-R45, 2001; M. Jenkinson, P. Bannister, M. Brady, and S. Smith, “Improved methods for the registration and motion correction of brain images,” Oxford University, Tech. Rep., 2002; and C. Penney, J. Weese, J. Little, D. Hill, and D. Hawkes, “A comparison of similarity measures for used in 2-D-3-D medical image registration,” IEEE Trans. on Medical Imaging, vol. 17, no. 4, pp. 586-595, 1998.
Some of the most widespread techniques are: histogram matching (J. Huang, S. Kumar, M. Mitra, and W. Zhu, “Spatial color indexing and applications,” in Proc. of IEEE Int'l Conf. Computer Vision ICCV '98, pp. 602-608); texture matching (J. Ashley, R. Barber, M. Flickner, D. Lee, W. Niblack, and D. Petkovic, “Automatic and semiautomatic methods for image annotation and retrieval in qbic,” in Proc. SPIE Storage and Retrieval for Image and Video Databases III, pp. 24-35); intensity cross correlation (J. B. Maintz and M. Viergever, “A survey of medical image registration,” Medical Image Analysis, vol. 2, no. 1, pp. 1-36, 1998); optical flow matching (M. Lef'ebure and L. Cohen, “Image registration, optical flow and local rigidity,” J. Mathematical Imaging and Vision, vol. 14, no. 2, pp. 131-147, 2001); kernel-based classification methods (N. Cristiani and J. Shaw-Taylor, Support Vector Machines and other kernel-based learning methods. Cambridge U. Press, 2000); and boosting classification methods (J. S. de Bonet and P. Viola, “Structure driven image database retrieval,” in Advances in neural information processing, vol. 10, 1997; T. Kieu and P. Viola, “Boosting image retrieval,” in IEEE Conference on Computer Vision and Pattern Recognition, 2000).
In such cases, it is well known that the standard linear cross correlation is a poor similarity measure. Previously, the images have been aligned based upon a single dimension feature, such as gray scale pixel intensity, using simple correlation techniques. These methods produce adequate results for images that are of a single modality. However, if the images are from different modalities or have high dimensional features, the single dimension feature approach does not produce adequate results.
Previous sequence alignment methods that can be applied to high dimensional data have been based on simple correlation measures that can handle the high computational load. When the sequence of images consists of only a few objects one can apply sophisticated methods that compute the entire empirical distribution of low dimensional features extracted from the sequence. These methods have superior performance when the features are well selected as they can adapt to non-linearities, spurious differences, and artifacts within the sequence. Examples include histogram equalization methods and methods based on minimizing joint entropy or maximizing mutual information (MI). However, these feature-distribution methods are difficult to apply as the dimension feature becomes high. Thus, these sequence alignment methods have been limited to low dimensional feature spaces, such as coincidences between pixel intensity levels.
The multi-modality images may arise from alignment of gene sequences over several genomes, across-modality registration of time sequences of patient scans acquired during cancer therapy, or simultaneous tracking of several objects in a video sequence. The previous sequence alignment methods fail when the objects in the sequence are high dimensional and are not exact replicates of each other, e.g., due to the presence of non intensity-preserving deformations or spurious noise and artifacts. In particular, computationally simple linear cross correlation methods fail to capture non-linear similarities between objects.
Entropic methods use a matching criterion based on different similarity measures defined as relative entropies between the feature densities. Entropic methods have been shown to be virtually unbeatable for some medical imaging image registration applications as discussed in C. R. Meyer, J. L. Boes, B. Kim, P. H. Bland, K. R. Zasadny, P. V. Kison, K. F. Koral, K. A. Frey, and R. L. Wahl, “Demonstration of accuracy and clinical versatility of mutual information for automatic multimodality image fusion using affine and thin-plate spline warped geometric deformations,” Medical Image Analysis, vol. 1, no. 3, pp. 195-206, April 1997 and D. Hill, P. Batchelor, M. Holden, and D. Hawkes, “Medical image registration,” Phys. Med. Biol., vol. 26, pp. R1-R45, 2001.
Several properties of entropic methods have contributed to their popularity for image registration: 1) because they are statistically based measures they easily accommodate combinations of texture based and edge based registration features; 2) relative entropies are easily defined that are invariant to invertible intensity transformations on the feature space; and 3) they are simple to compute and the number of local maxima can be controlled by suitably constraining the set of image transformations.
The difficulty in applying these entropic methods to a long sequence of images becomes almost insurmountable since the dimension of the feature space grows linearly in the length of the sequence. In order to apply these methods to long sequences, each pair of images has to be analyzed thereby increasing the likelihood of introducing computational errors. Further, such previous entropic methods have resulted in a computational bottleneck that is a major hurdle for information based sequence alignment algorithms.
These related art methods are characterized by one or more inadequacy. Specifically, limitations of the prior linear correlation methods do not extend to higher dimensional features and the prior entropic methods are equally unworkable with regard to higher dimensional features. Accordingly, it would be advantageous to provide a method that overcomes these inadequacies.